Towards a Rigorous Evaluation of Time-Series Anomaly Detection

In recent years, proposed studies on time-series anomaly detection (TAD) report high F1 scores benchmark TAD datasets, giving the impression of clear improvements in TAD. However, most apply a peculiar evaluation protocol called point adjustment (PA) before scoring. this paper, we theoretically and experimentally reveal that PA has great possibility overestimating performance; even random score can easily turn into state-of-the-art method. Therefore, comparison methods after applying lead to misguided rankings. Furthermore, question potential existing by showing an untrained model obtains comparable performance when is forbidden. Based our findings, propose new baseline protocol. We expect study will help rigorous further improvement future researches.